Towards Feature-Based Performance Regression Using Trajectory Data

Black-box optimization is a very active area of research, with many new algorithms being developed every year. This variety needed, on the one hand, since different are most suitable for types problems. But also poses meta-problem: which algorithm to choose given problem at hand? Past research has shown that per-instance selection based exploratory landscape analysis (ELA) can be an efficient mean tackle this meta-problem. Existing approaches, however, require approximation features significant number samples, typically selected through uniform sampling or Latin Hypercube Designs. The evaluation these points costly, and benefit ELA-based over default must therefore in order pay off. One could hope by-pass evaluations feature approximations by using samples would anyway perform, i.e., algorithm’s trajectory. We analyze paper how well such approach work. Concretely, we test accurately trajectory-based ELA approaches predict final solution quality CMA-ES after fixed budget function evaluations. observe loss predictions surprisingly small compared classical global approach, if remaining shall predicted not too large. Feature selection, contrast, did show any advantage our experiments rather led worsened prediction accuracy. inclusion state variables only moderate effect